Moments analysis in Markov reward models
نویسندگان
چکیده
We analyze the moments of the accumulated reward over the interval (0, t) in a continuous-time Markov chain. We develop a numerical procedure to efficiently compute the normalized moments using the uniformization technique. Our algorithm involves auxiliary quantities whose convergence is analyzed, and for which we provide a probabilistic interpretation. Key-words: Markov models, accumulated reward, performability, uniformization ∗ IRMAR IRISA † INRIA IRISA Analyse des moments dans les modèles markoviens à récompenses Résumé : Nous analysons les moments de la récompense cumulée sur l’intervalle (0, t) dans une châıne de Markov à temps continu. Nous développons une procédure numérique pour calculer efficacement les moments normalisés en utilisant la technique de l’uniformisation. Notre algorithme met en jeu des quantités auxiliaires dont la convergence est analysée et pour lesquelles nous fournissons une interprétation probabiliste. Mots-clés : Modèles markoviens, récompense cumulée, performabilité, uniformisation Moments analysis in Markov reward models 3
منابع مشابه
Analysis of Inhomogeneous Markov Reward Models
The majority of computational methods applied for the analysis of homogeneous Markov reward models (MRMs) are not applicable for the analysis of inhomogeneous MRMs. By the nature of inhomogeneous models only forward differential equations can be used to describe the model behaviour. In this paper we provide forward partial differential equations describing the distribution of reward measures of...
متن کاملAnalysis of the Completion Time of Markov Reward Models
Analysis of Markov Reward Models (MRM) with preemptive resume (prs) policy usually results in a double transform expression, whose solution is based on the inverse transformations both in time and reward variable domain. This paper discusses the case when the reward rates can be either 0 or positive, and analyses the completion time of MRMs. We present a symbolic expression of moments of the co...
متن کاملAnalysis of Markov Reward Models with Partial Reward Loss Based on a Time Reverse Approach
There are effective numerical methods for the analysis of Markov reward models (MRMs) without or with complete reward loss, but the analysis of MRMs with partial reward loss is more complex. This paper presents the analytical description of the distribution and the moments of the accumulated reward of partial increment loss reward models and an effective numerical method to evaluate these measu...
متن کاملAn Effective Approach to the Completion Time Analysis of On-off Markov Reward Models
Analysis of Markov Reward Models (MRM) with preemptive resume (prs) policy usually results in a double transform expression, whose solution is based on the inverse transformations both in time and reward variable domain. This paper discusses the case when the reward rates can be either 0 or a positive value c. These systems are called on-o MRMs. We analyze the completion time of on-o MRMs and p...
متن کاملNumerical Analysis of Large Markov Reward Models
First analysis of Markov Reward Models (MRM) resulted in a double transform expression, whose numerical solution is based on the inverse transformations both in time and reward variable domain. Better numerical methods were proposed based on the time domain properties of these models, such as the set of partial differential equation describes the process evolution in time. This paper introduces...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2007